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DEPART AD NT OF STATE 


Wasnugror, D.C. 20520 


State Dept. review 
completed 


MEMORANDUM 
STATINTL 
FROM: A/OPR - Virginia Schafer 


SUBJECT: PPG Tasking for Kecommendations to Improve 
Use of the Department's Automated Information 
System 


The Foreign Affairs Informetion System (previously referred 
to as Automated Document System) is a cornerstone for the 
Department's information future. It represents a modern, 
efficient Capability for utilizing information assets and 
for managing resources demended by the Department's foreign 
affairs information process - capabilities that few other 
agencies, outside of Defense and CIA, have available or under 
development. 

One of the greatest values of,this system is its potential 
use to offset human resources demanded by the Department's 
information process. This now totally manual process is the 
single most dominant and pervading of any in the Department. 
It consumes more time otf more people than any other single 
function. 


The system and its information provide a viable alternative to 
the present resource-consuming process of how information is 
used and maintained throughout the Department. As a viable 
alternative, appropriate application tem derive its benefits is 
the critical objective. The Department must move to a new 
plateau in utilization of =his valuable resource. It can 
reduce personnel burdens and concurrently sustain or increase 
the value of information by more effectively managing the 
information assets and resources involved. 


The continual need for botn additional financial and human 
resources demonstrates cleirly one significant fact: All 
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personnel resources are equalized at the common denominator 
level of position staffing. A messenger or clerical position 
counts against the Department's position ceiling equally as that 
of a Special Assistant, an Assistant Secretary, or an Ambassador 
position. 


The opportunity is available to make more effective use of the 
Department's scarce position resources by allocating them where am 
the benefits will be greater. By application of modern infor- 
mation technology, routine duties can be performed more 
efficiently and with less demand on human resources. Significant 
portions of an individual's time or entire positions can then 

be made available for more :mportant responsibilities and 
contributions. Freeing personnel time and positions from the 
mechanics of information handling is the objective of the 

Foreign Affairs Information System and its Central Foreign 

Policy File information resources. 


“At present, there are over 25,600 file cabinets in the Depart-— 
ment and another 22,300 at posts. These files contain over 
764 million pages of information materials. For a 5-year period, 
the Central Foreign Policy File contains approximately 11.8 
million pages of unique information. In a single year, 1979, 
over 62 million copies were made for primary distribution of 
some 927,000 cables and aircrams. This redundancy demands a 
substantial amount of human effort in existing manually 
operated paper systems. It represents a waste of human 
resources that can be avoided. Even with this unharnessed 
paper explosion, information is still fragmented and combina- 
tions of important signals are often lacking, or worse still, 
are submerged by massive quentities of tangential information. 


There is no efficient or efiective solution that can cope 
with present proportions of proliferated information in paper 
copies. The only solution is to prevent this proliferation 
without diminishing the accessibility of information. 

The Foreign Affairs Information System can provide this 
solution. 


To obtain maximum benefits trom the system, an investment will 
be required. This investment will minimize demands for per- 
sonnel by the paper handlina process and allow the Department to 
make better utilization of scarce human resources and more 
effective use of information to support the Department's mission. 


Briefly, recommendations for achieving these goals are: 


1. Demonstrate desirability and benefits of the automated 
resource by saturating a geographic bureau with terminals 
at the country desk and working officer levels. Terminals, 


or preferably word processors, should be linked through 
the information system to the Department's communications 
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resources so that cables, memoranda and other papers can 
be sent, delivered, received, and retrieved at terminals. 
This will satisfy the requirement for convenience and 
provide a prototype fcr identifying problems and solutions 
in moving toward an efficient office of the future. 


Convert airgrams to AIR CHANNEL cables and implement 
optical scanning of memoranda and other important papers. _ | 
This will increase comprehensiveness of the central 
resource, increase accessibility and thus reliability as 

an authoritative source of information. 


Improve capabilities cf the user language which provides 
access to the information resource, and invest in research 
to develop automated indexing from words in message text. 
These features will make the system more efficient and 
more accessible in previding required information, and 
minimize the friction between the user and the system. 


Provide the capability to identify papers to a case 

file and to reorganize and repackage search results to 

meet specific needs of users. This will increase responsive- 
ness of the system and make it a more viable tool in meeting 
information needs of users. 


A justification for these recommendations is presented in the 
attached analysis of impediments to maximum use of the system. 


Attachment: 

Constraints and Recommendations for Improving : 
Use of the Foreign Affairs Information System 

and Reducing Demands for Hunan Resources. 
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AND REDUCING DEMANDS FOR HUMAN RESOURCES 


In response to the PPG Tasking for recommendations to improve 
use of the Foreign Affairs Information System, the following 
analysis demonstrates the viability of solutions for impediments 
to more effective use of the Department's information system. — 


There is a major constraint to using this electronic or 
digital medium as a surrogate for paper files. It can best 
be characterized as the lack of system "accessibility". 
Terminals accessing the system are not located where they are 
needed and there are not enough terminals to compete with 
paper files. And, because they are accessible, "personal" 
copies of papers in office or working files are difficult to 
compete with even though their upkeep, use and disposition 
are unnecessarily demandin« of human resources. 


The lack of terminals where: they are needed, i.e. lack of 
convenience, is one of four factors that can inhibit or 
increase use of the system. The four key requirements of 
accessibility and use are: 


Oo Convenient - as ne«rby as a filing cabinet. 


o Comprehensive - complete in scope of relevant 
information. 
Oo Responsive - on recvest, provides the right infor- 


mation needed without missing pertinent information 
or without overburéening with irrelevant information. 


o Efficient - needed information can be found and 
used quickly and eesily with a minimum of effort. 


These requirements, and recommendations for meeting them are 
as follows: 


CONVENIENT ru 


The system must be as convenient as* paper office files to 
compete effectively with aré replace these files. 


A. CONSTRAINT 


The single greatest impedinent to accessibility through 
convenience is security anc its associated costs. A standard 
terminal for accessing the system costs between $2,000 and 
$4,000. It can be linked to the computer system with existing 
telephones in offices. However, since the system contains 
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classified information, special TEMPEST cleared (non-electro- 
magnetic radiating) terminals must be used. This raises the 
cost of terminals by a factor of 3 to $10,000 or $12,000. 

In addition, secure ‘conduit must be installed to link each 
terminal with the central computer system. Total price of 

an installed terminal is aporoximately $24,000. 


At this price, relatively f2w terminals have been installed 

in the Department. Most ar2 located adjacent to Assistant 
Secretaries' offices. However, a greater use of this facility 
can be made at the country desk and working officer levels 
where most of the paper files are located. To some 

extent, Assistant Secretaries do not need an information 
system - they already have one in the form of country desk 

and staff officers. The lazter require an information system 
- one that is as convenient es present paper files. 


B. SOLUTION ALTERNATIVES 


A secure telephone network is the most viable solution to this 
constraint. Another possib jity is the use of secure voice 
boxes to scramble or encrypt information over existing phone 
lines. This would allow anv telephone in the building to be 
used with a terminal to access the system. A third possibility 
is the use of fiber optics, in lieu of shielded conduit, to 
link terminals with the system. A fourth, less desirable, 
possibility is to separate unclassified from classified infor- 
mation and allow low cost terminal and telephone access to 
unclassified material. 


The latter alternative would produce a minimal benefit and 
thus, at best, a marginal solution. As put by one of the 
geographic bureaus, "It's the classified information that is 
of greatest interest and need. Access limited to unclassified 
information would be of minimum use." Fiber optics as an 
alternative would not substantially reduce costs over those 
presently incurred for shielded conduit. And there is still 
uncertainty as to whether or not fiber optics is vulnerable 

to penetration. 


At $10,000 per unit, secure voice boxes do not presently 

offer a substantial reducticn in cost. However, this 
alternative offers quick installation, and it is immune to 

the diminishing space required for installing conduit or fiber 
optics throughout the building. Lower cost versions of the 
secure box are supposed to ke available in 1982. The secure 
telephone network is some years away, and does not offer an 
immediate or short range solution. 
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C. RECOMMENDATION 


To prototype a "paperless", electronic office, one geographic 
bureau should be saturated at country desk level with terminals. 
Where possible, existing word processors should be used as 
terminals. This will avoid additional costs and proliferation 
of devices, each addressing a single facet of a multi-use 
information environment. These terminals, or word processors, 
will access the system to retrieve information from the central ~ 
file. One objective is to replace as many bureau files as 
possible. However, equally as important an objective is 
linking terminals, through =he computer system, with OC 
communications to allow for direct and paperless routing of 
cables for transmission, and for receipt of incoming cables. 
The same facilities can be used for delivery and receipt of 
memoranda. 


The use of existing word processors as terminals and cost 
and installation options for secure linkage should be examined 
by ISO and oc. 


Saturation of a bureau will provide a live working environment 
for identification of problems that must be resolved in 

moving toward an efficient office of the future. Paper will 
not be fully replaced as a means for delivering information. 
However, the retention of this information should be paperless. 


COMPREHENSIVE 


The automated central inforination resource can compete with, 
and become a viable replacement for, paper files if it is as 
complete and comprehensive as these office files. 


A. CONSTRAINT 


At present, the system is complete in its capture of cable 

and airgram traffic. However, it is lacking in important 
memoranda. This shortcoming is repeatedly demonstrated in 
response to Congressional oversight and FOI requests for 
information. It is highlighted by cases such as the Crown of 
St. Stephen, Moscow Radiation, and the recent Billy Carter- 
Libya Congressional inguiry. In all three instances, the 
preponderance of information resided’ in memoranda obtained 

from bureaus rather than from the Central Files. Answering 
requests, such as these, has imposed a search burden on bureaus, 
and has required more effor= to identify and discard duplicates 
than the original overlappine effort required for searching. 


Although bureaus are required to send copies of important 
memoranda and papers to FAIM for inclusion in the Central File, 
this policy and practice is not always observed. When they 
are sent in as paper media, they become microfilm records in 
the system, and the full text cannot be obtained on bureau 
terminals. Thus, there is no incentive for forwarding 
memoranda to FAIM for inclusion in the central resource. 
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To compete with paper files. these types of documents must 
be included in the central resource and they must be available 
in digital or electronic form, 


Also related to comprehensiveness is the duration of time in 
which information is available from the system. When the 
system was originally implemented, full text of cables were 
to be kept on line for 3 years. However, demands for system 
storage space have caused sable text to be removed from the 
System after 18 months. After this time period, message text 
can be obtained only from microfilm. And microfilm recovery, 
as it is structured on reel film in this application, is 
extremely inefficient and demanding of human effort. For 
example, a comparison of retrieval from the system and 
retrieval from microfilm identified a 7:1 difference in time 
and effort needed to retrieve 35 cables. As the number of 
messages increases, the ef<ort required for retrieval from 
microfilm reels increases at a disproportionate rate. 


This human resource demand problem is more apparent in FAIM 
than in user bureaus. As an information service center, FAIM 
must frequently locate information dating back to the early : 
1960's. Microfilm, in this application, is only marginally 
more efficient than paper jiles. 


B, SOLUTION ALTERNATIVES 


There are two approaches available for obtaining information 

in memoranda and other paper media: (1) through word processors, 
or (2) through optical scarning of papers. Both will provide 
information in digital or electronic form. 


The first alternative woulé require linking existing word 
processors with the system. These devices are used to prepare, 
create, and originate the Gocuments in question. The system 
could automatically capture these information materials in 
electronic form directly from the source. However, as 
previously mentioned, the Single impediment to this type of 
linked network is the cost of conduit or other secure linkage. 


The second alternative would require that memoranda and 

other papers be prepared in a specified format, and in an 
optically recognizable typewriter font. Acquisition by the 
system would not be automatic as with direct capture from 

word processors. Originators would have to forward paper 
copies to FAIM where they would be optically scanned and input 
into the system. This would make information materials avail- 
able to bureau terminals and provide an immediate incentive 

for bureaus to include them in the central information resource. 
A more comprehensive resourcze will provide greater accessibility 
to information. 
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C. RECOMMENDATION 


FAIM is installing an optical scanner to capture these types 
of papers for the post information protection program. This 
program will provide properly organized microfiche copies of 
cables, airgrams (AIR CHANNEL cables), memoranda, and other 
essential documents to replace paper files at high risk posts. 
The scanner will also be used to capture similar information 
originated within the Department. A new memorandum form is 
currently being designed for testing this process. 


With a more comprehensive central resource, the system will be 
more responsive and of more: value to user needs. When an 
appropriate level of comprehensiveness is attained, bureaus 
will no longer be required to search office files in response 
to Congressional and FOI recuests. At that point the Central 
File will have replaced the: necessity for consulting office 
Files. 


In relation to comprehensiveness as it concerns availability 
over time, ISO is continuing its investigation of low cost, mass 
storage. Recent advancements in Very Large Scale Integrated 
(VLSI) circuit technology are drastically reducing costs and 
increasing storage capacit:es by orders of magnitude. VLSI 
chips, half the size of a domino, can store millions of 
Characters of information. Low cost technology will soon offer 
storage for extended time periods, and thus increase comprehen- 
Siveness and accessibility in this domain. 


EFFICIENT . 


Effort required to identifv and find needed information must 
be minimized by an efficient, powerful user query language 
and a more flexible vocabu_ary of subject terms. 


A. CONSTRAINT 


One form of the system's inefficiency can be characterized as 
the experience and knowledve required of the user. This, 
however, is not a major deficiency or constraint of the system. 
Users must learn the query Janguage as‘well as the thesaurus 
(controlled vocabulary of words) used for indexing subject 
content of communications and documents. The query language 
could be improved by providing simpler, multi-functional 
commands and by combining several related tasks into a single 
command. Although efficiency in the user language or commands 
is already acceptable, refinement would make usage more 
efficient and more desirab_e. 


The system indexing vocabu_ary does require more user knowledge. 
There is no response from ihe system to key words or subject 
terminology that are not in the vocabulary. The correct terms 
must be used. 
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B. SOLUTION ALTERNATIVES 


Changes and improvements in the user command language are already 
being defined by FAIM. However, only minimal changes can be 

made to the present system before such changes become a major 
system redesign effort. 


As to the vocabulary, the use of automated indexing, to select 
nouns, qualifiers, and proper names from message text, will 
produce precision indexing and consequential efficiencies in 
retrieval. Automated indexing must be accompanied by synonym 
linkage since this method of indexing is prey to the vagaries 
of language occurring in message text. Combined automated 
indexing and synonym linkage would in essence provide free 
retrieval from free indexing. With any subject word entered 
by a user, all related words would be shown by the system to 
facilitate maximum recall cf pertinent information. Retrieval 
of information will then be based on natural language. 


A third alternative is the use of "private" indexing, i-.e., 
indexing by originators and recipients of cables and memoranda. 
Present indexing by FAIM can be considered "public" in nature 
since it is performed for a broad range of users. Private 
indexing will supplement the information resource with the 
user's perspective of important subjects. 


The goal of all of these fcrms of indexing is to make retrieval 
of needed information not cnly possible but relatively simple. 
The retrieval task must be made as natural to the user as 
possible. Simplicity and efficiency of retrieval are dependent 
on substantial organizing end indexing of information. However, 
easy and effective retrievel is illusive because language is 
dynamic -- words and their meanings change. Indexing finding 
aids must also be refined to keep pace with dynamics of the 
reporting language used by Foreign Service Officers. 


C. RECOMMENDATION 


FAIM currently is encouraging private indexing by users. 

It is being performed on a limited basts by the USUN and by 
some posts. To increase the value of the post microfiche 
program (to replace paper tiles), posts will be given 
instructions and will be encouraged to index messages they 
originate. 


FAIM is also developing specifications for automated indexing 

and synonym linkage. To date, neither industry nor government 

has developed a system and methodology that performs satisfactorilly. 
The problem is not computer related. It is an intellectual 
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problem dealing with lanauaqe, ambiguities in word relationships, 
and the ability to recover specific as well as conceptual level 
subject information. Addi-ional financial resources will be 
required to bring the best expertise (consultants) to bear on 


The system must deliver ali of the right information when 
demanded, and it must present this information in an organized 
Structure that facilitates its use, 


A. CONSTRAINT 


Responsiveness involves sone of the principles of efficiency. 
The system must deliver ali information relevant to a subject 

~ not miss important information. It must also deliver the 
right information, and not Overload the requester with 
tangential or irrelevant irformation. In this respect, indexing 
must be in-depth and of hich quality. The better the quality 

of indexing, the less effort required to identify, find, and 
retrieve required information. 


The present quality of indexing will provide an acceptable 
level of responsiveness. However, the FAIM staffing level is 
inadequate for the volume of communications that must be 
indexed ona timely basis. {nformation cannot be retrieved 
if messages have not been indexed, and as a consequence the 
resource suffers from lack of comprehensiveness. The value 
in being current and comprenensive has been weighed against 
the value of quality, in-denth indexing, i.e., not being able 
to find information versus ~-he potential for finding it with 
increased effort. As a resilt, FAIM has opted for quick, 
shallow indexing to keep current. This surface indexing has 
diminished quality to the extent that broader, general searches 
are required to find specif c information. Consequently 

the proportion of non-relevant to relevant information in a 
search response is greater +han it should be. Solutions to 
this. constraint are included with those to make system use 
more efficient. : 


Another more serious constr¢int to responsiveness is presented 
by the inability to restructure or sort “retrieved information 
into a logical, usable order. This is serious where large 
quantities of information are retrieved for a single request. 
Examples are the People's Temple case (over 1,000 documents), 
Libya-Billy Carter case (over 2,500 documents), and special 
Tehran analysis (over 17,00C cables). In each of these 
cases, as well as numerous cthers, retrieved paper copies of 
cables had to be manually scrted in an order that would 
facilitate use and analysis. The effort required has been 
substantial - 2 to 4 people for up to 8 days depending on 
quantity. 
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This inability to sort and reorganize information also 
diminishes the potential for replacing office paper files. 
In L, for example, there are over 270 file cabinets. 
Portionsof these files deal with extradition cases, anda 
sampling indicates that approximately 80 percent of their 
contents consists of cable traffic. These types of paper 
files would be ideal candicates for the central system, 
But the inability to organize documents by case, and then 
organize within a case, nullifies system potential for 
replacing paper files. 


B. SOLUTION 


Provide users with a Capability to identify documents to a 
case file and to reorganize and repackage search results 
to meet specific needs. 


C. RECOMMENDATION 


Modify the present system t>. allow for user initiated sorting 
of key features identifying the characteristics of messages 
and their content. These kav features should include date, 
originator, addressee, security classification, subject terms, 
personal names, TAGS, case »r file number, etc. In conjunction 
with recommendations for imoroving the user query language 

and subject terminology for access, the need for a new system 
should be investigated. This need will be predicated on 

the ability to implement changes to the present system and 
the impact of these changes on the operational integrity of 
the system, 


These recommendations should be accorded a main thrust effort 


with the best expertise ava’ lable. Each recommendation will 
require financial investment and some marginal need of human 
resources for a short time period. The investment can benefit 


the Department by providing the basis to better manage the 
utilization of scarce human resources and more effectively use 
the Department's information assets. 
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